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Detailed Action 



Drawings 

Objections 

1 . Though the drawings permit examination, they are informal and clearly are unacceptable. The drawings 
are objected to for at least the following reasons. 

a. The drawings are of such poor quality that they do not permit adequate reproduction. 

b. Captions are generally illegible (e.g. Figs. 4a and Fig. 5). 

c. Fig. 7b does not adequately illustrate the corresponding description in the specification (e.g. page 
7, lines 14-15 and page 13, lines 1-3 of the Applicant's Specification). Fig. 7b shows two arbitrary 
intersecting lines not the set of tracked pixels within an N x N pixel neighborhood. Proper 
labeling of this drawing may rectify this matter. 

d. A line appears to extend from top left comer of plane 302 to the lower right comer of plane 302 in 
Fig. 8. This line is barely visible. Left this way, Fig. 8 becomes confusing. 

e. Fig, 13 is particularly abhorrent. It is not clear what is being shown in Fig. 13. It does not appear 
that faces are being shown as indicated on page 16, lines 9-10 of the Specification. 

f According to page 16, line 1 1 of the Applicant's Specification, regions 0,4 and 0,5 are filled. Fig. 
13 seems to show region 4,5 as filled. 

g. The vertices and edges of Fig. 13 and Fig. 14 seems to correspond to one another according to the 
second paragraph on page 16 of the Applicant's Specification. This is not apparent from Fig. 14. It 
is suggested that the process illustrated in Fig. 14 be labeled in such a way as to distinguish it from 
the filling process illustrated in Fig. 13 (and, correspondingly change the specification), or redraw 
Figs. 13 and 14 so that the correspondence between these two figures is clear. 

h. It is not clear what the dashed line from vertex 0 to vertex 2 in Fig. 14 represents (Perhaps it 
indicates that face 0,1,2,V folds or consists of two triangles). 
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i. Fig. 14 contains apparently extraneous edges - i.e. the series of unlabeled edges connecting vertex 

0 with vertex 2. 

Generally speaking, the drawings (not flow diagrams), several of which illustrate processes in 3 dimensions admit to 
some indication of viewpoint or perspective (e.g. Figs. 8, lOb-d, and 13-14). 

2. The drawings are objected to under 37 CFR 1.83(a) because they fail to show state 900 (page 16, line 9 of 
specification) as described in the specification. Any structural detail that is essential for a proper understanding of 
the disclosed invention should be shown in the drawing. MPEP § 608.02(d). A proposed drawing correction or 
corrected drawings are required in reply to the Office action to avoid abandonment of the application. The objection 
to the drawings will not be held in abeyance. 



Specification 

Objections 

3. The disclosure is objected to because of the following informalities. 

a. On page 3 line 13 of the Applicant's disclosure, the applicant refers to Patent No. 6,187,392. 
Patent 6,187,392 discloses a laser desorption of CVD precursor species, which clearly is not 
related to the applicant's invention. 

b. On page 5, line 9 of the Applicant's disclosure, "one or vertices" should be corrected to read, "one 
or more vertices". 

c. On page 10, lines 12-13 or the Applicant's Disclosure, the applicant states, "The process of 
projecting a three-dimensional point onto the image plane in a given fi-ame is referred to as 
projection". Given Fig. 3, the process of projection according to the Applicant's claimed invention 
is more specifically referred to as perspective projection. 

d. On page 13, line 3 of the Applicant's disclosure, the applicant refers to a "feature point's central 
pixel". Indicating that feature points have a central pixel implies that they have some spatial 
extent. This is contrary to the defmition of feature points, 3D or 2D (e.g. "[t]he entries of the 
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feature point data correspond to the coordinate positions in each image which a true 3D feature 
point is viewed" - Applicant's Abstract - and "3D feature points are tracked to determine 2D 
feature points" - page 5, lines 3-5 of the AppHcant's specification), 
e. On page 13, lines 17-18 of the Applicant's disclosure, the applicant states, "Alternatively, for each 
vertex, "V," a plane is fitted to the entire Hst of nearest neighbors of V and a normal for that plane 
is calculated". Alternatively implies that deriving the normal by fitting a plane to the nearest 
neighbors, in the manner just stated, is done in place of the other method (i.e. "V and all its nearest 
neighbors, "nn," are projected onto a 2D plane 302. If at least two of these projected nearest 
neighbors exist, these nearest neighbors and V are connected to form a triangle 304. For each 
triangle 304, the cross product is used to calculate a normal vector, for example, normal vectors 
306a, 306b, 306c, and 306d. These normals are robustly averaged to determine a normal for V" - 
Applicant's specification, page 13, second paragraph) and vice versa. However, the Applicant 
goes on to state, "If the normals computed fi-om the above two approaches agree sufficiently, the 
normal firom the second method is used. Otherwise, the operation is flagged for fiirther processing. 
In such a case, the normal calculated firom the first method is used". Therefore, both methods are 
used in determining the normal. Replacing the word alternatively with additionally would yield a 
more consistent description of this process. 
Appropriate correction is required. 

Claims 

Objections 

4. The disclosure is objected to because of the following informalities. In claim 1 (line 8), "positions of a true 
3D features" should be changed to "positions of true 3D features". In claim 15, "pint clouds" should be replaced 
with "point clouds". Appropriate action is required. The objected claims will be treated as if those changes were 
made. 
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Rejections Under 35 U.S.C. § 1 \2(2) 

5. Claim 1-26 are rejected under 35 U.S.C. 112, second paragraph, as being indefinite for failing to 
particularly point out and distinctly claim the subject matter which applicant regards as the invention. 

6. Claims 1-26 claim a method for nearest neighbor selection to construct a 3D model from a sequence of 2D 
images of an object or scene. However, the claims seem more directed toward a method for constmcting a 3D model 
from a sequence of 2D images of an object or scene. In particular, notice that the steps in claim 1 of providing a set 
of images, tracking features, generating depth data, aligning depth data, connecting vertices and using visibility 
information to arbitrate vertex connections are related to building a 3D model not nearest neighbor edge selection, 
especially when one realizes that there are no edges to select prior to the step of connecting vertices. Henceforth, the 
claims will be treated as though they are claiming a method for constructing a 3D model from a sequence of 2D 
images of an object or scene, as opposed to a method for nearest neighbor edge selection. It is understood, however, 
by the Examiner that the applicant intends the inventive feature of the claimed 3D modehng method to be the way in 
which nearest neighbor edges are selected. 

7. Furthermore, in regard to claims 6-7, it is not clear to what central vertex the applicant is referring. In the 
remainder of this document, claim 6 will be interpreted as, "The method of claim 5, further comprising building a 
nearest neighbors list that specifies a set of candidate connections for each vertex, V, the nearest neighbors being 
other vertices that are visibly near the vertex V and claim 7 will be interpreted as "The method of claim 6, further 
limiting the near neighbors list to vertices that are close, in 3D, to the vertex F". 

8. Lastly, according to claim 13, ..no candidate edge can occlude the view of that face in that view if the 
face is determined to be completely visible in any original view...". Stated as such claim 13 can be confusing. No 
face can be expected to be completely visible in any viewpoint. Claim 13 seems to constrain the visible face to be 
visible in all viewpoints. A more valid statement of claim 1 3 could be, "The method of claim 9, wherein for each 
candidate surface face, when the face is a polygon or surface patch bounded by three candidate model edges chosen 
from a set of near neighbor lists, no candidate edge can occlude the view of that face in that view if the face is 
determined to be completely visible in any of the original views, and any such occluding edge is pruned from the 
near neighbor lists". Claim 13 will be interpreted in this manner henceforth in this document. 
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Rejections Under 35 U.S.C. § 102(e) 

9. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the 

rejections under this section made in this Office action: 

(e) the invention was described in (1) an application for patent, published under section 122(b), by another filed in the United States 
before the invention by the applicant for patent or (2) a patent granted on an application for patent by another filed in the United States 
before the invention by the applicant for patent, except that an international application filed under the treaty defined in section 351(a) 
shall have the effects for purposes of this subsection of an application filed in the United States only if the international application 
designated the United States and was published under Article 21(2) of such treaty in the English language. 

10. Claims 1-4, 16-18, 22-24 and 26 are rejected under 35 U.S.C. 102(e) as being anticipated by Davidson et 
al. (U.S. Patent 6,516,099). 

1 1 . The following is in regard to Claim 7. Davidson et al. disclose an image-based modeling apparatus and 
method for constructing a three-dimensional (3D) model of an object from images taken from a plurality of images 
taken of the object from a plurality of arbitrary viewpoints - i.e. a sequence of 2D images of the object (Davidson et 
al. Abstract). This method (and apparatus) comprises the steps of: 

(La.) Providing a set of images from different views of the object or scene (see, for example, 
Davidson et al. Fig. 2). 

(l.b.) Tracking features of the scene among successive images (e.g. steps S4-S6 in Davidson et 
al. Fig. 3). Using the tracked features to establish correspondence between the 2D 
coordinate positions of true 3D features as viewed in each image. See the discussion in 
Davidson et al. relating to. step SIO of Davidson et al. Fig. 3. The details of step S 10 are 
illustrated in Davidson et al. Figs. 6-19. 

(I.e.) Generating depth data of the features of the scene from each image of the sequence. Note 
that by generating 3D data (i.e. Davidson et al. Fig. 3, step SIO) the method of Davidson 
et al. necessarily entails generating depth data, since one of the three dimensions is a 
dimension indicating depth. Similarly, the depth data corresponds to the coordinate 
position of the feature along a depth axis for each image. This is illustrated in Davidson 
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et al. Figs. 33 and 35. There it should be clear that depth data per image (viewpoint) is 
the distance from the (approximate) intersection - e.g. point 68 in Davidson et al. Fig. 35 
- of the projected rays (e.g. rays emanating from image planes 50 in Davidson et al. Fig. 
35) to the corresponding image point on the respective image planes. According to 
Davidson et al. (Davidson et al. column 24, lines 63-67), depth is measured as a distance 
from a camera image plane for that image view, 
(l.d.) Aligning the depth data in 3D to form vertices of the model. This process is illustrated in 
Fig. 41 of Davidson et al. This alignment is accomplished by projecting corresponding 
points in the sequence of images in 3D. See Davidson et al. column 40, lines 52-67 to 
column 41, lines 1-10. 

(l.e) Connecting the vertices to form the edges of the model (step S12 in Davidson et al. Fig. 

3) - i.e. by Delaunay triangulation. See Davidson et al. Fig. 49 and column 46, lines 57- 
65. 

(l.f) Using visibility information from feature track data, original images, depth data and input 
edge data to arbitrate among multiple geometrically feasible vertex connections to 
construct surface detail of the 3D model. See steps 592-602 in Davidson et al. Fig, 49. 
Visibility is determined by projecting rays from the image planes (analogous to the 
original images) of the various viewpoints (e.g. LI . . .L5 in Davidson et al. Fig. 3), 
through feature points in the image planes to the corresponding points in 3D space (see 
Faces and, therefore, edges (derived or "input" from the previous step) are eliminated 
based on this visibility. The result is a 3D triangulated mesh representing the surface of 
the object. See Davidson et al. column 47, lines 6-31 and lines 45-51. 
Note that the method of Davidson et al. and the applicant's claimed method can be generally classified as image- 
based modeling methods. It has thus been shown that the method of Davidson et al. is a 3D modeling method 
comprising, at least, all steps claimed in Applicant's claim 1. Therefore, the teachings of Davidson et al. anticipate 
the method set forth in Applicant's claim 1 . 
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12. The following is in regard to Claim 2. As shown above, Davidson et al. teach a method of 3D modeUng in 
accordance with claim 1. Davidson et al further disclose the step of tracking includes identifying 2D feature points 
from the images of true 3D feature points (Davidson et al. Fig. 3, step S4 and column 6, lines 58-62), and 
establishing correspondence of the 2D feature points among a set of images (e.g. steps S62 and S64 in Davidson et 
al. Fig. 7 and Davidson et al. column 10, lines 9-12), to generate a 2D feature track. In Davidson et al, feature tracks 
are represented as the set of matching feature points, in the set of images corresponding to different viewpoints. Also 
note that the fundamental matrix derived in step S250 of Davidson et al Fig. 25 (see Davidson et al column 24, 
lines 1 1-30) indicates mathematically the correspondence between the 2D feature points of any two images in the 
sequence of images. Therefore, the teachings of Davidson et al address the limitations of claim 2. 

13. The following is in regard to Claim 3. As shown above, Davidson et al teach a method of 3D modeling in 
accordance with claim 2. It should be clear that, in the method of Davidson et al, depth data and 2D feature points 
are projected into a common 3D world coordinate system (i.e. 3D space in which the object is assumed to reside). 
See, for example, Davidson et al Fig. 29a. Therefore, the teachings of Davidson et al anticipate the method set forth 
in claim 3. 

14. The following is in regard to Claim 4. As shown above, Davidson et al. teach a method of 3D modeling in 
accordance with claim 3. The method of Davidson et al. further comprises generating a point cloud for each feature 
point from the 3D projection, with each entity of the point cloud corresponding to the projected 2D feature point 
from a respective image. Refer to the discussion in column 41, lines 46-67 to column 42, lines 1-14 of Davidson et 
al See also Fig. 43, In this way, the method of Davidson et al conforms to that which is set forth in claim 4. 

15. The following is in regard to Claim 16. As shown above, Davidson et al teach a method of 3D modeling in 
accordance with claim 1. In the method of Davidson et al, the depth data is measured as the distance from a 
corresponding 3D feature point to the camera image plane for a given camera view of the true 3D feature point. This 
is evident from Davidson et al Fig, 35 and the discussion in Davidson et al column 41, lines 15-23. In this way, the 
method of Davidson et al is in accordance with the claimed method of claim 16. 

16. The following is in regard to Claim 17, 22-24 and 26. As shown above, Davidson et al teach a method of 
3D modeling in accordance with claim 1. The depth data is obtained from the input sequence of images. In this 
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manner, the input sequence of images can be regarded as input depth data. Interpreted in this way, the teachings of 
Davidson et al. anticipate the method of claim 17. 

17. A similar line of reasoning can be applied to each of claims 22-24 and 26. 

18. The following is in regard to Claim 18. As shown above, Davidson et al. teach a method of 3D modeling in 
accordance with claim 1. The depth data generated according to the above step (l.c) of Davidson et al.'s method can 
be regarded as intermediate, in the sense that it is data generated at an intermediate step between the input of the 
sequence of images and the output of the polygonal 3D model. In this way, the teachings of Davidson et al. 
anticipate the method of claim 17. 



Rejections Under 35 U.S.C. § 103(a) 

19. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all obviousness rejections set 
forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102 of this title, if 
the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would 
have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

20. Claim 5 and 14 are rejected under 35 U.S.C. 103(a) as being unpatentable over Davidson et al. in view of 
Chotiros (U.S. Patent 4,891,762). 

21 . The following is in regard to Claims 5, As shown above, Davidson et al. teach a method of 3D modeling in 
accordance with claim 1 . For each of the point clouds, Davidson et al. consolidates the point cloud into a one vertex 
(i.e. Davidson et al. Fig. 48 step S584). Davidson et al. use statistical methods to remove extraneous points from the 
point cloud, under observation, and to combine remaining points into a single vertex (Davidson et al. Figs. 44-48). 
Davidson et al., however, do not show or suggest consolidating the point cloud into one or more vertices, each 
vertex representing a robust centroid of a portion of the point cloud. 

22. Chotiros disclose a method for tracking features. As observed by Chotiros, several measured data points of 
a 3D feature point may be spread spatially within a confidence interval indicative of the uncertainty of the feature 
point's location (Chotiros column 6, lines 46-61). Note that this confidence interval is analogous to the error 
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ellipsoid of Davidson et al. (Davidson et al. column 44, lines 42-53) and the point clouds of both the Applicant and 
Davidson et al. The error ellipsoid is essentially a probability distribution indicating the potential locations of a 
given 3D feature point. Similarly, point clouds essentially define regions where the potential 3D feature points may 
exist. Chotiros replaces the groups of data points corresponding to 3D feature points in scene by a single 
representative point at the centroid of the group (Chotiros column 6, lines 67-68). 

23. The teachings of Davidson et al. and Chotiros are combinable because they are analogous art, in the sense 
that both disclose methods involving feature tracking. Therefore, it would have been obvious to one of ordinary 
skill in the art, at the time of the applicant's claimed invention, to consolidate each point clouds by representing them 
by the their respective centroid, as opposed to consolidating them by the statistical means proposed by Davidson et 
al. The motivation/suggestion for doing so would have been to eliminate the unnecessary redundancy inherent to the 
point cloud, as suggested by Chotiros (Chotiros column 6, lines 61-63). Furthermore, consolidating using the 
centroid is a computationally simpler approach to representing the point cloud than the statistical methods proposed 
by Davidson et al. The teachings of Davidson et al. and Chotiros, when combined in the manner just described, 
demonstrate a 3D modeling method that satisfies all limitations of claim 5. 

24. The following is in regard to Claim 14. As shown above, Davidson et al. teach a method of 3D modeling in 
accordance with claim 4. Davidson et al. do not, however show or suggest that this method should be such that the 
step of using includes consolidating the point cloud into one or more vertices, each vertex being located within a 
convex hull of the point cloud and satisfying visibility criteria for each image in which the corresponding true 3D 
feature is visible. 

25. According to the discussion above, relating to claim 5, Chotiros consolidates each point cloud into a 
centroid representing that point cloud. The convex hull of a set of points contains all elements of that set. This fact 
can be used to show that a centroid of a set of points is contained in the convex hull of those points. Thus, 
consolidating a cloud of points into a single vertex represented by the centroid of those points ensures that this 
vertex lies within the convex hull of those points. 

26. According to the teachings of Davidson et al., these points must satisfy some visibility criteria for each 
viewpoint in which the corresponding 3D point is visible. See the discussion below with regard to claim 8. 
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27. The teachings of Davidson et al. and Chotiros are combinable for the same reasons given above with regard 
to claim 5. The motivation for combining these teachings is also the same as given above. Furthermore, ensuring 
vertices adhere to the visibility criteria, as taught by Davidson et al., allows concave surfaces to be modeled as well 
as unnecessary interior faces to be removed from the model (Davison et al. column 46, lines 65-67 to column 47, 
lines 1-5. Davidson et al.). Therefore, it would have been obvious to one of ordinary skill in the art, at the time of the 
applicant's claimed invention, to consolidate each point clouds by representing them by the their respective centroid, 
as opposed to consolidating them by the statistical means proposed by Davidson et al. By doing so, one obtains a 
method that conforms to the limitations of claim 14, 

28. Claim 6-9 and 11-13 are rejected under 35 U.S.C. 103(a) as being unpatentable over Davidson et al. in 
view of Chotiros, in further view of Boissonnat ("Geometric Structures for Three-Dimensional Shape 
Representation", ACM Transactions on Graphics, 1984). 

29. Thefollowing is in regard to Claim 6. As shown above, the teachings of Davidson et al. and Chotiros can 
be combined to satisfy all the limitations of claim 5, Neither Davidson et al. nor Chotiros expressly mention building 
a nearest neighbors list that specifies a set of candidate connections for each vertex, V the nearest neighbors being 
other vertices that are visibly near the vertex V. 

30. As stated above, Davidson et al. use Delaunay triangulation to connect the derived set of vertices, thereby 
forming a polygonal mesh representing the object or scene depicted in the sequence of images. Davidson et al. omit 
the details of this triangulation. 

3 1 . Boissannat proposes an algorithm for the Delaunay triangulation of an unorganized set of 3D points 
(analogous the set of derived vertices of Davidson et al.'s method since initially no known structure, [besides basic 
assumptions about the topology of the object or scene's surface] or connectivity is assigned to these points). As 
described by Boissannat, the Delaunay triangulation associates with each point Mi a set (or list) of at least three 
neighbors, Mj, which are the nearest neighbors of Mi in different directions. See last paragraph on page 273 of 
Boissannat. Though not explicitly defined by Boissannat, it is clear that the algorithm contains some structure 
wherein for each vertex Mi of the set of vertices a corresponding set of nearest neighbors Mj is assigned. Each such 
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Mj is connected to the vertex Mi by an edge. See Fig. 8 and first sentence of the last paragraph on page 273 of 
Boissannat. This structure will be referred to here as the nearest neighbors list (L). 

32. The teachings of Davidson et al., Chotiros and Boissannat are combinable since the method of Davidson et 
al. uses Delaunay triangulation on a set of unorganized pomts, namely the set of derived vertices (i.e. the centroids 
of the aforementioned point clouds). Therefore, it v^ould have been obvious to one of ordinary skill in the art, at the 
time of the applicant's claimed invention, to implement the Delaunay triangulation employed in Davidson et al.'s 
method according to teachings of Boissannat. The algorithm of Boissannat is attractive as a means for Delaimay 
triangulation, since according to Boissannat (Boissannat page 278, last paragraph), in practical cases the algorithm 
has an almost linear conputational complexity (i.e. almost 0{N)). Combining the teachings of Davidson et al, 
Chotiros and Boissannat, in the manner described above, jdelds a method in accordance v^th claim 5, further 
comprising building a nearest neighbors list that specifies a set of candidate connections for each vertex, the nearest 
neighbors being other vertices that are visibly near the said vertex. 

33. The following is in regard to Claim 7. As shown above, the teachings of Davidson et al., Chotiros, and 
Boissannat can be combined to satisfy all the limitations of claim 6. As noted above with respect to claim 6, the 
nearest neighbors are the vertices nearest (in 3D, of course, since vertices are from 9^^) to the vertex in question (e.g. 
Af/). Therefore, combining the teachings of Davidson et al., Chotiros and Boissannat, in the manner described above, 
yields a method that satisfies the limitations of claim 7. 

34. The following is in regard to Claim 8, As shown above, the teachings of Davidson et al, Chotiros, and 
Boissannat can be combined to satisfy all the limitations of claim 6. In both the algorithm of Boissannat and the 
method of Davidson et al, faces are removed from the Delaunay triangulation when they fail to satisfy a visibility 
criteria. See Section 3.2 of Boissannat - specifically, the last paragraph of page 276 to the second paragraph on page 
278 (step (c)) - and Davidson et al. column 47, lines 5-51. The removal of faces corresponds to removal of edges 
between nearest neighbors and, hence, a pruning of the nearest neighbor list. According to Davidson et al., this is 
done to eliminate spurious and interior faces (Davidson et al. column 47, lines 3-4). In this way, combining the 
teachings of Davidson et al., Chotiros and Boissannat, in the manner described above, yields a method that satisfies 
the limitations of claim 8. 
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35. The following is in regard to Claim P. As shown above, the teachings of Davidson et al., Chotiros, and 
Boissannat can be combined to satisfy all the limitations of claim 8. Davidson et al. removes faces from the 
Delaunay triangulation that do not satisfy some visibility criteria. Davidson et al. accomplishes this by casting a ray 
from the camera to a 3D point on the surface of the model (i.e. a vertex) and if that ray intersects a surface (i.e. if the 
surface occludes the 3D point) that surface is removed from the triangulation. This is repeated for each of the 
viewpoints. See Davidson et al. column 47, lines 5-51. Here, the 3D point represents trusted edge data in a similar 
vein as the applicant because at least one edge containing that 3D point must be visible from at least one viewpoint. 
The visibility of this point is known a priori since it was derived from a 2D feature observed from at least one 
viewpoint. Therefore, in the method obtained by combining the teachings of Davison et al., Chotiros, and 
Boissannat, as discussed above, candidate edges and faces for the model are tested for visibility against trusted edge 
data. 

36. The following is in regard to Claim 11. As shown above, the teachings of Davidson et al., Chotiros, and 
Boissannat can be combined to satisfy all the limitations of claim 9. As discussed above, the derived vertices (3D 
points) of the method of Davidson et al. can be reasonably interpreted as trusted edge data. These vertices can be 
regarded as 3D edge data (see the discussion above with regard to claim 6). Therefore, trusted edge data (i.e. the 3D 
vertices) can be trivially derived from 3D edge data, namely the 3D vertices. When interpreted in this way, trusted 
edges are derived from 3D edge data in the method obtained by combining the teachings of Davidson et al, 
Chotiros, and Boissannat, in the manner discussed above. 

37. The following is in regard to Claim 12 As shown above, the teachings of Davidson et al., Chotiros, and 
Boissannat can be combined to satisfy all the limitations of claim 9. As discussed above, the derived vertices (3D 
points) of the method of Davidson et al. can be reasonably interpreted as trusted edge data. As discussed above, 
these vertices are derived from depth data, which can be interpreted as depth edge data, since knowing the depth of 
the two vertices connected by an edge gives the depth of all points along that edge. Taken in this way, trusted edges 
are derived from depth edge data (i.e. the depth data used to derive the vertices) in the method obtained by 
combining the teachings of Davidson et al., Chotiros, and Boissannat, in the manner discussed above. 

38. The following is in regard to Claim 13. As shown above, the teachings of Davidson et al., Chotiros, and 
Boissannat can be combined to satisfy all the limitations of claim 9. In Delaunay triangulation, faces of triangulation 
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of the vertices are, of course, triangles (i.e. surface faces that are polygons bounded by three model edges chosen 
from a set of near neighbors. Recall that a set of near neighbors is given for each vertex in the list L - see above). 
According to Davidson no surface (defined by three edges) should obscure any face determined to be visible in a 
given viewpoint. Surfaces (and thus the edges defining these surfaces) that violate this criterion are removed. See 
Davison et al. column 46, lines 65-67 to column 47, lines 1-5, Davidson et al. do this so that faces, obtained by 
Delaunay triangulation, are removed when they cover concave regions of the surface model. Therefore, the 
teachings of Davidson et al. address the limitations of claim 13. In this way, in the method obtained by combining 
the teachings of Davidson et al., Chotiros, and Boissannat conforms to all limitations of claim 13. 



39. Claim 10 is rejected under 35 U.S.C. 103(a) as being unpatentable over Davidson et al. in view of Chotiros 
and Boissonnat, as applied to claim 9, in further view of Debevec ("Modeling and Rendering Architecture from 
Photographs", 1996). 

40. The following is in regard to Claim 10. As shown above, the teachings of Davidson et al, Chotiros, and 
Boissannat can be combined in such a way as to satisfy all limitations of claim 9. However, neither Davidson et al., 
Chotiros, nor Boissannat teach that the trusted edge data should be derived from silhouette edge data. 

41. Debevec discuss several methods for the automated construction of 3D models of scenes or objects from 
observation of those objects or scenes. One such method discussed is "shape from silhouette contours". See section 
on pages 10-15 of Debevec. These contours represent silhouette edge data. Furthermore, these contours indicate 
which portions of the object or scene's surface are visible from each of the viewpoints. See, for example. Figure 2.3 
on page 14 of Debevec. In this way, the silhouette contours can be regarded as trusted edge data in the same vein as 
the Applicant's trusted edge data. Both are edges (corresponding to visible regions of the object of scene's surface) 
that are known to be visible a priori from some viewpoint, 

42. The teachings of Debevec, relating to silhouette contours, are combinable with teachings of Davidson et al. 
because both are analogous art. Both are related to deriving a 3D model representing an object depicted in images 
obtained by observing the object from several viewpoints. Therefore, it would have been obvious to one of ordinary 
skill in the art, at the time of the appHcant's claimed invention, to derive the trusted edge from the silhouette 
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contours or simply use these contours as the trusted edge data, when determining the visibility of edges and faces in 
the method of Davidson et al. The motivation to do so would be that the silhouette contour provides a simple 
structure (conceptually envision a "cookie-cutter" - see the caption of Debevec Fig. 2.3 on page 14) against which 
the shape (e.g. the profile of the convex hull obtained by the Delaunay triangulation) of the object or scene, in each 
view, can be compared. Surfaces of the model that do not correspond to the silhouette can be easily identified and 
quarantined ("cookie-cutting"). Combining the teachings of Davidson, Chotiros, Boissannat and Debevec in this 
manner yields a method that conforms to the method set forth in claim 10. 

43. Claims 19, 21 and 25 are rejected under 35 U.S.C. 103(a) as being unpatentable over Davidson et al. in 
view of Debevec. 

44. The following is in regard to Claim 19. As shown above, Davidson et al. disclose a method that conforms 
to limitations of claim 1. Davidson et al., however, fail to teach that the depth data is obtained from a laser sensing 
system. 

45. Debevec discuss several methods for the automated construction of 3D models of scenes or objects from 
observation of those objects or scenes. One such method discussed is range-scanning which involves directly 
obtaining depth information from scanning the object or scene of interest with a laser and laser sensors. See section 
2.8 on page 18 of Debevec. 

46. The teachings of Debevec related to laser range scanning are compatible with those of Davidson et al since 
both are concerned with constructing models of an object of scene from observed or derived depth data. Therefore, it 
would have been obvious to one of ordinary skill in the art, at the time of the applicant's claimed invention, to use 
depth data obtained from laser range-scanning, in lieu of deriving that data from various 2D images, in the method 
of Davidson et al. Clearly, the motivation to do so would have been to obviate the more cumbersome process of 
computationally deriving the depth data. Combining the teachings of Debevec with those of Davidson et al., in this 
manner, yields a method that conforms to the hmitations of claim 19. 

47. The following is in regard to Claim 2L As shown above, Davidson et al. disclose a method that conforms 
to limitations of claim 1. Since lasers often operate in the infra-red portion of the electromagnetic spectrum, the laser 
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ranging systems inherently represent IR-based sensing systems. Therefore, the discussion above relative to claim 19 
is also applicable to claim 21. 

48. The following is in regard to Claim 25. As shown above, Davidson et al. disclose a method that conforms 
to claim 1. Davidson et al., however, fail to teach providing silhouette edge data as input data. 

49. Debevec discuss several methods for the automated construction of 3D models of scenes or objects from 
observation of those objects or scenes. One such method discussed is "shape from silhouette contours". See section 
on pages 10-15 of Debevec. These contours represent silhouette edge data. These are input to derive the surface of 
the object or scene. See, for example. Figure 2.3 on page 14 of Debevec. 

50. Davidson et al. and the teachings of Debevec relating to silhouette contours are combinable because they 
are analogous art. Both are related to deriving a 3D model representing an object depicted in images obtained by 
observing the object from several viewpoints. Therefore, it would have been obvious to one of ordinary skill in the 
art, at the time of the applicant's claimed invention, to use the silhouette edge data as input edge data in the method 
of Davidson et al. Clearly, this has the advantage of obviating the need for the computationally expensive Delaunay 
triangulation. The teachings of Davidson et al. and Debevec (relating to silhouette contours), when combined in the 
manner just described, yield a method that conforms to the method set forth in claim 25. 



5 1 . Claim 20 is rejected under 35 U.S.C. 103(a) as being unpatentable over Davidson et al. in view of Fenster 
et al. (U.S. Patent 6,342,891). 



52. The following is in regard to Claim 20. As shown above, Davidson et al. disclose a method that conforms 
to limitations of claim 1. Davidson et al., however, fail to teach that the depth data is obtained from a sonar sensing 
system. 

53. Fenster et al. teach that in three-dimensional ultrasound imaging systems when a succession of two- 
dimensional images have been captured and digitized, the two-dimensional images are stored as a stack to form an 
image data array. Before a three-dimensional image of the scanned volume can be created and viewed by a user, the 
image data array must be reconstructed to form a volumetric image array. This type of reconstruction, in which 
every pixel in every two-dimensional image slice is converted into an appropriate voxel in an image volume (i.e. 
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volumetric image array) prior to display is known as "full volume" reconstruction (Fenster et al. column 1, lines 40- 
50). Voxels are discrete 3D points and therefore constitute depth data. Ultrasound systems are essentially sonar 
sensing systems. 

54. These teachings of Fenster et al. are combinable with Davidson et al. because Fenster et al. disclose a 
means for obtaining depth data and the method of Davidson et al. uses depth data. Therefore, it would have been 
obvious to one of ordinary skill in the art, at the time of the applicant's claimed invention, to use depth data obtained 
from a 3D ultrasound imaging system (as taught by Fenster et al.), in lieu of deriving that data from various 2D 
images, in the method of Davidson et al. Clearly, the motivation to do so would have been to obviate the more 
cumbersome process of computationally deriving the depth data. Combining the teachings of Davidson et al. with 
those of Fenster et al. yields a method that conforms to the method set forth in claim 20. 



Allowable Subject Matter 

Objections, Allowable Subject Matter 

55. Claim 15 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten 
in independent form including all of the limitations of the base claim and any intervening claims. 

56. The following is a statement of reasons for the indication of allowable subject matter. 

57. No prior art was found that addressed or could be combined to address the limitations of claim 15 - that is, 
a method m accordance with claim 4, wherein the step of using includes projecting a set of pint clouds into a 
multitude of shared views, a shared view being an original image view that contributes 2D feature points to each 
point cloud in the set, and projecting vertices derived from each point cloud in the set into the shared views, and the 
step of using requires the 2D arrangement of the projected vertices, in each shared view, being consistent with the 
2D arrangement of the contributing 2D feature points from that view. 
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58. For each point in a point cloud, Davidson et al.'s method ensures the consistency with the 2D arrangement 
of the contributing 2D feature points from that view. See Figs. 34-35 and column 34, lines 45-67 to column 35, lines 
1-26 of Davidson et al. Points in 3D deemed to correspond to the same feature point (i.e. the triple of corresponding 
points - Davidson et al. Fig. 34) are projected into a multitude of views, each of which contain the corresponding 2D 
feature point in their respective images. See Fig. 35 of Davidson et al. Consistency is measured by observing 
whether or not the projected point (e.g. point 70 in Fig. 35 of Davidson et al.) is within a threshold distance of the 
actual 2D feature point (e.g. point 72 in Fig. 35). The triple of corresponding points is saved if this is case (step 436 
in Davidson et al. Fig. 34), Stored triples of corresponding points later become the points of the point cloud. In this 
manner, Davidson teaches the step of using includes projecting a set of pint clouds mto a multitude of shared views, 
a shared view being an original image view that contributes 2D feature pomts to each point cloud in the set and the 
step of using requires the 2D arrangement of the projected vertices, in each shared view, being consistent with the 
2D arrangement of the contributing 2D feature points from that view. The method set forth in claim 15 distinguishes 
itself from the method obtained by the combination of Davidson et al., Boissannat and Chotiros by including the 
step of projecting the derived vertices into the shared views. While Davidson et al.'s method projects points 
constituting the point clouds into shared views (prior to derivation of the point clouds and vertices), they do not 
suggest, nor do Boissannat and Chotiros, the projection of vertices. Vertices are, in general, not points of the point 
clouds but, rather, representations thereof 



Citation of Relevant Prior Art 

59. The prior art made of record and not relied upon is considered pertinent to applicant's disclosure: 
[1] U.S. Patent 6,208,347. Migdal et al. 

[2] U.S. Patent 6,473,079. Kacyra et al. 

[3] Surface Reconstruction from Unorganized Points. Ph.D. Thesis. 1994. Hugues 
[4] A New Voronoi-Based Surface Reconstruction Algorithm. 1998. Amenta et al. 

60. [l]-[4] generally disclose methods of constructing 3D models from arbitrary sets of 3D points. 
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